Augmented Reality-based Feedback for Technician-in-the-loop C-arm Repositioning

Interventional C-arm imaging is crucial to percutaneous orthopedic procedures as it enables the surgeon to monitor the progress of surgery on the anatomy level. Minimally invasive interventions require repeated acquisition of X-ray images from different anatomical views to verify tool placement. Achieving and reproducing these views often comes at the cost of increased surgical time and radiation dose to both patient and staff. This work proposes a marker-free "technician-in-the-loop" Augmented Reality (AR) solution for C-arm repositioning. The X-ray technician operating the C-arm interventionally is equipped with a head-mounted display capable of recording desired C-arm poses in 3D via an integrated infrared sensor. For C-arm repositioning to a particular target view, the recorded C-arm pose is restored as a virtual object and visualized in an AR environment, serving as a perceptual reference for the technician. We conduct experiments in a setting simulating orthopedic trauma surgery. Our proof-of-principle findings indicate that the proposed system can decrease the 2.76 X-ray images required per desired view down to zero, suggesting substantial reductions of radiation dose during C-arm repositioning. The proposed AR solution is a first step towards facilitating communication between the surgeon and the surgical staff, improving the quality of surgical image acquisition, and enabling context-aware guidance for surgery rooms of the future. The concept of technician-in-the-loop design will become relevant to various interventions considering the expected advancements of sensing and wearable computing in the near future

Pelphix: Surgical Phase Recognition from X-ray Images in Percutaneous Pelvic Fixation

Surgical phase recognition (SPR) is a crucial element in the digital transformation of the modern operating theater. While SPR based on video sources is well-established, incorporation of interventional X-ray sequences has not yet been explored. This paper presents Pelphix, a first approach to SPR for X-ray-guided percutaneous pelvic fracture fixation, which models the procedure at four levels of granularity -- corridor, activity, view, and frame value -- simulating the pelvic fracture fixation workflow as a Markov process to provide fully annotated training data. Using added supervision from detection of bony corridors, tools, and anatomy, we learn image representations that are fed into a transformer model to regress surgical phases at the four granularity levels. Our approach demonstrates the feasibility of X-ray-based SPR, achieving an average accuracy of 93.8% on simulated sequences and 67.57% in cadaver across all granularity levels, with up to 88% accuracy for the target corridor in real data. This work constitutes the first step toward SPR for the X-ray domain, establishing an approach to categorizing phases in X-ray-guided surgery, simulating realistic image sequences to enable machine learning model development, and demonstrating that this approach is feasible for the analysis of real procedures. As X-ray-based SPR continues to mature, it will benefit procedures in orthopedic surgery, angiography, and interventional radiology by equipping intelligent surgical systems with situational awareness in the operating room

Retrieval of Salt Marsh Above-ground Biomass From High-spatial Resolution Hyperspectral Imagery Using PROSAIL

Salt marsh vegetation density varies considerably on short spatial scales, complicating attempts to evaluate plant characteristics using airborne remote sensing approaches. In this study, we used a mast-mounted hyperspectral imaging system to obtain cm-scale imagery of a salt marsh chronosequence on Hog Island, VA, where the morphology and biomass of the dominant plant species, Spartina alterniflora, varies widely. The high-resolution hyperspectral imagery allowed the detailed delineation of variations in above-ground biomass, which we retrieved from the imagery using the PROSAIL radiative transfer model. The retrieved biomass estimates correlated well with contemporaneously collected in situ biomass ground truth data ( R2=0.73 ). In this study, we also rescaled our hyperspectral imagery and retrieved PROSAIL salt marsh biomass to determine the applicability of the method across spatial scales. Histograms of retrieved biomass changed considerably in characteristic marsh regions as the spatial scale of the imagery was progressively degraded. This rescaling revealed a loss of spatial detail and a shift in the mean retrieved biomass. This shift is indicative of the loss of accuracy that may occur when scaling up through a simple averaging approach that does not account for the detail found in the landscape at the natural scale of variation of the salt marsh system. This illustrated the importance of developing methodologies to appropriately scale results from very fine scale resolution up to the more coarse-scale resolutions commonly obtained in airborne and satellite remote sensing